Skip to content

[FLINK-40087][mysql] Fix MySqlDefaultValueConverter fails to parse charset-introducer default values wrapped in double quotes and/or containing padding spaces (e.g. _UTF8MB4" 0 ")#4464

Open
gong wants to merge 3 commits into
apache:masterfrom
gong:mysql-ddl-default-value
Open

Conversation

@gong

@gong gong commented Jul 7, 2026

Copy link
Copy Markdown
Contributor

FLINK-40087 Fix MySqlDefaultValueConverter fails to parse charset-introducer default values wrapped in double quotes and/or containing padding spaces (e.g. _UTF8MB4" 0 ")

…arset-introducer default values wrapped in double quotes and/or containing padding spaces (e.g. `_UTF8MB4" 0 "`)
@gong

gong commented Jul 7, 2026

Copy link
Copy Markdown
Contributor Author

@leonardBang @yuxiqian PTAL

@yuxiqian yuxiqian left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One minor comment: can we add tests for snapshotting tables with double quoted default value fields?

IIRC default values are parsed differently in snapshotting (by SHOW CREATE TABLES) and incremental stages (by parsing DDL).

gong added 2 commits July 10, 2026 19:13
…ntroducer default values wrapped in double quotes and/or containing padding spaces (e.g. `_UTF8MB4" 0 "`)
…ntroducer default values wrapped in double quotes and/or containing padding spaces (e.g. `_UTF8MB4" 0 "`)
@gong

gong commented Jul 10, 2026

Copy link
Copy Markdown
Contributor Author

One minor comment: can we add tests for snapshotting tables with double quoted default value fields?

IIRC default values are parsed differently in snapshotting (by SHOW CREATE TABLES) and incremental stages (by parsing DDL).

@yuxiqian Thanks for the review! I added testSnapshotWithCharsetIntroducerDefaultValue which creates a table with both single-quoted (_utf8mb4' 0 ') and double-quoted (_UTF8MB4" 0 ") character-set-introducer defaults (covering TINYINT / INT / BIGINT / DECIMAL / DATE / DATETIME / TINYINT(1)) before the Flink source starts, so the schema is discovered via SHOW CREATE TABLE during the snapshot phase.
While writing this I probed the actual output of SHOW CREATE TABLE on MySQL and found that the server normalizes those default expressions on the write path — it strips the introducer, strips the padding inside the quotes, and rewrites double quotes to single quotes. So the snapshot path never actually observes the raw shape and cannot serve as a reverse reproduction for the bug. The new case therefore acts as a positive verification that the fix does not introduce any regression on the snapshot schema discovery path (source reaches RUNNING, snapshot rows arrive, and a subsequent binlog INSERT relying on server-side defaults also flows through). The reverse reproduction remains covered by testAlterWithCharsetIntroducerDefaultValue which feeds the raw ALTER TABLE DDL through the binlog. I documented this in the Javadoc of the new test.

@gong gong requested a review from yuxiqian July 10, 2026 11:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants